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ABSTRACT 



The returns on continuing vocational training (CVT) in 
enterprises were examined through a review of research on the returns on 
employer-provided training. A search of "Econ Lit," ERIC, and other databases 
identified more than 90 publications on the topic. Twenty-one publications 
examining the relationship between training inputs and training outputs from 
a variety of perspectives were selected for in-depth analysis . The following 
were among the review's main conclusions: strict rates of return have rarely 
been calculated; training is shown to have positive effects on wages and 
productivity; evidence suggests that training received from one employer 
increases productivity and wages with another employer; and selection effects 
matter in training. The studies were also analyzed from the standpoint of the 
issues involved in measuring the returns to CVT investments . The amount of 
empirical research concerned with estimating returns to CVT was discovered to 
be relatively sparse and underdeveloped. It was recommended that more thought 
be given to methodological issues, such as the suitability of qualitative and 
quantitative research and whether they can complement one another. (The 21 
studies are summarized in an annotated bibliography. The document contains 93 



references.) (MN) 
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This report contains a review of research undertaken within and beyond the European 
Union to identify and measure the returns to continuing vocational training (CVT) in 
enterprises. The report is not written as an end in itself, but rather as an input to a longer 
term CEDEFOP project to undertake further research in this area. Increasing investment 
in enterprise CVT is one aspect of achieving the notion of ‘lifelong learning for all’, but 
the general lack of information as to the returns of such training for the state, sectors, 
enterprises, and individuals means that there are a number of difficulties in evaluating 
current and future investment decisions. This information ‘gap’ is not easy to fill, partly 
due to insufficient methodological tools and data, but also due to certain conceptual 
constraints: for example, defining CVT, given that it is a generic term encompassing a 
number of different training activities. 

The aim of this review is, therefore, to discuss a number of these difficulties and 
summarise how existing studies have approached them as a means to assess how existing 
work might be developed or supplemented. While the original parameters for this review 
were wide, the scope of the existing research turned out to be narrower than initially 
expected; hence as a review the report concentrates on available work, much of which has 
been done in the U.S. It is worth noting that many of the studies are very recent (post 
1990 ), which indicates a growing interest and ability to explore the returns to enterprise 
CVT. 

Alan Barrett, an economist from the Economic and Social Research Institute (ESRI) in 
Ireland, and Ben Hovels, a sociologist from the Instituut voor Toegepaste Sociale 
Wetenschappen (ITS) in the Netherlands prepared this report as a preliminary stage in 
which it is intended to stimulate debate on how additional research in this area could best 
be undertaken. The disciplinary mix of the participants ensured that a broad range of 
research was included. 

In order that this report represent a true discussion document, CEDEFOP would be very 
interested in hearing other views concerning existing or future research projects into the 



returns to enterprise CVT. In addition, we would welcome your views on this report and 
your suggestions for future work. For this reason, a questionnaire accompanies this report 
as a means to gain additional insights and ideas from a number of other sources. On this 
basis, participation in this work could be extended so as to formulate and execute a 
research project. 



If you would like a copy of this questionnaire, please do not hesitate to contact: 

Sarah Elson-Rogers/Sven-Age Westphalen 
CEDEFOP 
P.O. Box 27 (Finikas) 

GR-55102 

Thessaloniki 

Greece 

E-mail : ser@cedefop.gr/saw@cedefop.gr 
Fax: 00 30 31 490 117 

The deadline for comments is the end of November 1998. 

We would like to thank you for giving this first report your attention and look forward to 
receiving your comments and suggestions in the near future. 



Sarah Elson-Rogers/Sven-Age Westphalen 
Project co-ordinators 



Stavros Stavrou 
Deputy Director, CEDEFOP 
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1.1 Background 

This review presents research which has attempted to estimate the returns to employer- 
provided training, discusses the conceptual and methodological issues which have arisen 
in this context, and suggests ways in which research in this area can be usefully expanded. 
Before getting into the detail of the review, it is important to explain why these sire 
important issues and why additional research effort should be employed. 

In a world of growing international competition and accelerating technological change, 
firms face a variety of pressures and challenges related to training. Competition requires 
that costs be contained; all areas of possible containment, including training, must be 
assessed. Competition also requires, however, that the workforce of each firm be as 
productive as possible, thus creating a need for increased training expenditure; 
technological change generates similar pressures. It is this tension between the reasons for 
increasing and reducing training at the level of the firm that produces the need for a 
rigorous understanding of the returns to training. Without reliable information on the 
relationship between the costs and benefits of training, decisions on the financing of 
training must be made on the basis of hunches and impressions. In such a setting, the 
potential for the misallocation of resources to training is large, for example, through 
insufficient investment in training or through excessive spending on certain types of 
training. 

In a similar manner, competitive pressures have required those within firms who are 
responsible for training to prove the benefits of training in terms of the effect on the 
“bottom line”. Just as firms increasingly buy in services rather than provide them from 
within, there are growing pressures to hire in employees with the required skills, even for 
very short periods, as opposed to training existing staff and providing them with the skills. 
As a result, the nature of the employment relationship is changing with an increase in, for 
example, short-term contracts. In such a situation, the long-term goals of employers and 
employees may differ given that it is less likely that they will have a long-term 
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relationship. Employees who are transitory will want their training to be general in nature 
so that they can transfer it easily. The work of Becker (1975) reviewed below indicates 
that this makes it more likely that firms will not finance training; hence, the burden is 
more likely to fall on employees themselves or the state. 

While the need for estimating the returns to training may seem clear, the question can still 
be asked as to why such estimation should not be left to individual companies. 
Alternatively, why is it important for social science researchers to explore this issue? 
There are a number of dimensions to the answer. If firms feel under pressure to 
economise on their training expenditure, they are even more likely to face such pressures 
in relation to devoting resources to training evaluation. Hence firms may not undertake 
the evaluations'. But even if they do, independent social scientists still have a role. First, 
it is rare that individual companies publish evaluation results, so any information 
generated tends not to be disseminated. Second, and more importantly, evaluations done 
by individual firms will yield results which are specific to the firm itself. It requires the 
social scientist’s expertise to generate results which are more generally applicable through 
the use of standardised techniques in the study of a larger range of companies. 

1.2 Research to date 



It is only in recent years that researchers have begun to make attempts at measuring the 
returns to employer-provided training. Although they have been estimating the returns 
to more formal schooling for many years, returns to employer-provided training were 
thought to be captured by estimating the link between work experience and wage 
growth. Just as wages were seen to be higher for individuals with more schooling, so 
also were wages seen to be higher for individuals with more work experience and longer 
tenure in a particular job. The interpretation put on this finding was that workers 
receive training in jobs and become more productive over time. By looking at how 
wages rose with experience and making assumptions on the costs of training, rates of 
return could be estimated. 



1 This point is returned to in Section 3.1 below. 
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The assumed link between the eamings-experience profile and training came to be 
questioned in the 1970s and 1980s from both theoretical and empirical perspectives. 
One theoretical question was raised by Lazear (1979). He argues that wages rise with 
experience because employers want to provide workers with an incentive to remain with 
the firm; if higher wages are postponed to later in the worker’s career, the worker will 
be induced to remain with the employer. If this is true, then wages are not necessarily 
linked with experience and productivity, as is implied by the training argument. On the 
empirical front, the paper by Medoff and Abraham (1981) raised further questions about 
the link between experience and productivity. Drawing on the personnel files of a large 
firm, they were able to demonstrate that although more experienced workers were paid 
more, they were no more productive. 

Given the questions that had been raised by these authors and others, it was clear that an 
important avenue of research was the direct estimation of the benefits of training. As 
will be seen below, the lines of enquiry followed have included estimating the link 
between training and wages (as opposed to experience and wages), the link between 
training and productivity, and the effects of training on other variables such as reducing 
the incidence of unemployment and increasing job satisfaction. Studies have also begun 
to differentiate between different forms of training, for example, off-the-job and on-the- 
job. 



1.3 Methodology for this review 

As with all research in the social sciences, the studies whose quality has been validated 
through peer-review can be found in scientific journals, and so it was primarily to these 
journals that we turned in looking for research on the topic. We have, however, also 
drawn on working papers which allow us to consider the most recent work. In searching 
for studies we began by looking through the relevant records in the CD-ROM version of 
Econ Lit, which is the most comprehensive reference collection for journal articles and 
books in economics, and through relevant data-files such as ERIC. We also consulted 
colleagues who provided other references, in particular references to studies as yet 
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unpublished. Finally, we studied recent reviews which touched on the issues of interest 
here. 

It should be noted that at the outset of the literature search, the definition of ‘returns’ 
being employed was wide, i.e. we were looking at the returns to enterprise training for 
individuals, the enterprise and the state on the assumption that this incorporated both 
training effects and training effectiveness. It can be seen that many of the studies 
reviewed below have looked at returns only in terms of the ‘effects’ of training on 
productivity and wage growth rather than the effectiveness of the training process itself, 
and the review reflects this. 

It should also be noted that this review, due to a lack of information, does not address in 
a detailed way the issue of whether returns differ according to who funds employer- 
provided training. We know that in some countries enterprise training receives a high 
level of state subsidy, while in others, individuals are expected to co-invest in training 
with the enterprise. At this stage, we are merely concerned with studies which have 
attempted to establish a causal link between the provision of training within the 
enterprise and a ‘return’, as a means to assess who should fund training. 

The content and structure of this review has therefore been determined to a large degree 
by the content of the studies included, the scope of which turned out to be narrower than 
expected. As mentioned, we had intended to include studies which looked at returns to 
individuals, firms and the state. The state perspective, however, is particularly 
underdeveloped, and the review reflects this by focusing its discussion on the returns for 
the enterprise and the individual. While the impact of education on growth rates across 
countries has been addressed in the economics literature, very little work has been done 
on employer-provided CVT. This in itself is an interesting finding. 
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In the first part of this section, we present outlines of a range of studies which have 
looked at the relationship between training inputs and training outputs from a variety of 
perspectives. Rather than arranging the studies into categories, we have decided to 
present the studies initially in a chronological fashion. In the second part of the section, 
we will draw out the conclusions of the studies, and at this point a categorisation of sorts 
will emerge. 

2.1 The studies 

Booth, Alison L. (1991), "Job-Related Formal Training: Who Receives It and 
What is It Worth?", Oxford Bulletin of Economics and Statistics, Vol. 53, No. 3, pp. 
281-294. 

As the title suggests, Booth’s interest was in establishing who received training and, 
more importantly for current purposes, what the wage gain was for those who trained. 
The usefulness of drawing on a large-scale survey of individuals to explore the 
relationship between training inputs and outputs is seen in the study. Booth’s data 
comes from the British Social Attitudes Survey of 1987. This is a representative sample 
of the entire population; however, only employees from this sample are included in the 
analysis, leaving 1,365 individuals. The individuals in the survey are asked to provide 
information on the amount of training they have received. However, the data used by 
Booth contain a weakness. Training in the survey is broken down by ‘formal’ and 
‘informal’ training, with ‘formal’ loosely relating to a structured approach to training 
and ‘informal’ including activities like watching others doing the job. Clearly this is a 
useful distinction, but some of the usefulness is lost because formal training is reported 
by the individual in days undertaken while informal training is only reported as having 
been undertaken or not. Booth finds a significant relationship between training and 
wages, with the effect being particularly evident for women. 
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Lynch, Lisa (1992), "Private-Sector Training and the Earnings of Young 
Workers", American Economic Review, Vol. 82, No. 1, pp. 299-312. 

Lynch (1992) begins her study of the link between training and wages by making the 
following observation: 

Due to a lack of appropriate data, researchers have been unable to examine 
directly the impact of private sector training on wages in any comprehensive 
way. Consequently, many have had to infer this impact from the shape of wage 
profiles, (p 299) 



She overcomes this difficulty by drawing on a survey conducted in the United States, the 
National Longitudinal Survey youth cohort (NLSY). In this survey, 12,686 males and 
females who were 14-21 years of age at the end of 1978 were interviewed every year 
starting in 1979; Lynch uses the data up to 1983. Extensive information on their 
training activities is collected. In particular, individuals report how many weeks they 
had spent in on-the-job training, off-the-job training (such as in vocational colleges) and 
in apprenticeships. Information on wages is also included. 

Such data allows Lynch to do two things. First, she can work out how individual 
characteristics influence the likelihood of receiving training. As this is not our primary 
interest here, we will look at her second task, that is estimating how different forms of 
training effect wages. She finds, among other things, that off-the-job training acquired 
with either the current or previous employers increases wages. She does this by 
estimating regression equations, with the log of wages as the dependent variable. 
However, on-the-job training with a previous employer does not increase wages. The 
difference in the impact of on- and off-the-job training with a previous employer may be 
because the ‘on’ may be specific to that employer whereas the ‘off may be more 
general. It could also be that current employers are only prepared to recognise off-the- 
job training from a previous employer because the content is more transparent. 
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Holzer, H., R. Block, M. Cheatham, and J. Knott (1993), “Are Training Subsidies 
for Firms Effective? The Michigan Experience”, Industrial and Labour Relations 
Review Vol. 46, No. 4, pp. 625-636. 

Unlike the previous two studies which looked at training benefits from the perspective 
of the individual, this study looks at training from the perspective of the firm. The data 
used arose out of a grant programme run by the State of Michigan, through which grants 
were made available to manufacturing companies for the financing of training. By 
surveying companies which had received grants, and others who had applied but did not 
receive a grant, a data set was generated with information on training inputs and 
companies’ outputs. Hence, the authors were able to explore whether there was a link 
between the two, working with samples of between 171 and 250 firms. 

One particularly useful feature of the data is that it provides information on the 
companies over a number of years. In attempting to link training and productivity this is 
important. With data for one year, any observed relationship between training and 
productivity must be treated with caution. If there is an aspect of the firm which is 
correlated with training and which also increases productivity, but which is not observed 
in the data, then the effect of training may be overstated. For example, if management 
quality is high in a firm, this may lead to greater training and to greater productivity. A 
statistical analysis with one year of data will miss this point and will attribute the 
management effect on productivity to training, thus overstating the effect of training. 
With data for more than two years, this problem is reduced somewhat. If the 
unobserved characteristics of the firm do not change over time, it is possible to look at 
how productivity changes across firms are related to changes in training. 

Holzer et al. use the scrappage rate 1 as a measure of productivity and hours of training 
per employee as their measure of training. They find that increased training reduces the 
scrappage rate; for example, a doubling of the amount of training per employee reduces 
the scrappage rate by about 7 percent. Hence, they find evidence of a direct link 
between training and productivity. 



1 The “scrappage rate” is the proportion of units manufactured that must be discarded due to faults. 
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Van Sandick, A.S., and A.M. Schaap-Neuteboom (1993), Het rendement van een 
bedrijfsopleiding: Een instrument voor het bepalen van het financiele rendement van 
trainingen, Groningen: RUG. 

This study involved the estimation of the profitability of training for managers in a large 
Dutch supermarket. As all managers underwent training, it was not possible to conduct 
a treatment-and-control-type study. Instead, the approach adopted was to compare 
managers, who underwent training, with others who performed the same tasks as 
managers on occasions, but who were not managers and thus did not undergo training. 
The effectiveness of the training was evaluated by means of interviews with the 
employees in question, more senior managers and other employees. 

The most interesting aspect of the study for current purposes involves the manner in 
which the perceived effects of training were translated into monetary amounts. It is a 
general theme of this review that putting monetary values on the effects of training is 
not a simple matter, and as such, the approach used here may point to a possible 
pragmatic approach to overcoming this difficulty. In essence, the effects of training for 
each worker are estimated on a number of 5-point scales. Scores for each worker are 
calculated, and assumed monetary values are attached to them. There is clearly an ad 
hoc element in the selection of the monetary values, but the approach does allow for a 
range of estimates. It can be argued that the range can be too large to be useful and this 
is certainly an argument that must be taken seriously. 

Bartel, A. (1994), “Productivity Gains from the Implementation of Employee 
Training Programs”, Industrial Relations , Vol. 33, No. 4, pp. 411-425. 

Bartel (1994) looks at the link between training and productivity using another survey of 
employers, the Columbia Business School survey. Like Holzer et al., her data covers a 
number of years, so it is possible to relate changes in training input to changes in 
productivity. Her measure of productivity is based on sales. Her training measure is 
somewhat weak in that it is the proportion of workers trained. She does find a positive 
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effect of training, although interpreting the effect is difficult because of the nature of her 
training variable. 

Bishop, John H. (1994), "The Impact of Previous Training on Productivity and 
Wages", in Lisa Lynch (ed.), Training and the Private Sector : International 

Comparisons, Chicago: University of Chicago Press. 

A different approach to testing for the training/wage link is taken by Bishop (1994). 
Whereas other studies such as Booth (1991) and Lynch (1992) used large-scale surveys 
of individuals, Bishop uses two surveys of employers conducted in the United States. 
The first is the Employment Opportunity Pilot Project - National Centre for Research in 
Vocational Education (EOPP-NCRVE) survey which was undertaken in 1982. 
Although 3,412 owners/managers were interviewed for this survey, only 480 were of 
use to Bishop in this study due to the particular nature of the analysis. The second 
survey was one conducted by the National Federation of Business; questionnaires were 
mailed to around 1 1,000 firms, of which 2,599 responded. For his analysis, Bishop was 
able to use 2,285. 

In these surveys, amongst other things, the employers were asked to provide information 
on two of their employees, in particular information on their training, wages and 
productivity. Bishop then compares the experiences of the two workers across a large 
number of firms. In order to explore the link between training and the outcomes of 
interest, Bishop constructs a ‘training-time index’; he does this by taking the amount of 
time reported in different forms of training and multiplying it by the cost of that time. If 
a trainer is involved in the training, this cost is added to the cost of the trainee’s time. In 
general, Bishop finds that training increases wages and productivity, but he draws 
attention to two other findings. First, the increase in productivity is greater than the 
increase in wages, so training is profitable for employers. Second, training received 
from a previous employer increases wages, but again, it increases productivity more. 
Hence, there is a ‘spillover’ from employer-provided training, in that the training 
provided by one employer benefits another. 
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Groot, Wim, Joop Hartog and Hessel Oosterbeek (1994a), "Returns to Within 
Company Schooling of Employees: The Case of the Netherlands", in Lisa Lynch 
(ed.), Training and the Private Sector: International Comparisons, Chicago: 

University of Chicago Press. 



Groot et al. (1994a) 2 draw attention to an important issue in the estimation of training 
effects, that is the fact that training is likely to be acquired by those who will use it best. 
It may be that employers will chose to train certain workers or because certain workers 
will volunteer for training; either way, there will be a selection process. The importance 
of this point in estimating the effect of training on wages is that if the observed effect is 
based on those who were selected for training, and not randomly assigned to training, 
the effect for the random individual will be over-stated. Using Dutch data, Groot et al. 
employ a particular statistical technique to take account of the selection issue. They 
show that while the effect of training on the wages of those who participate is positive, 
there would not have been a positive effect for those who do not participate, had they 
trained. 3 



Diederen, J. (1994), Opleidingsniveau en heroepsloophaan: Over de effecten van 
initiele opleiding en additionele scholing op de heroepsloophaan van lager 
opgeleiden, Nijmegen: ITS. 



This study is based on a large-scale survey of individuals. The researchers had available 
to them a longitudinal dataset based on a group of people who left primary education in 
1965; the most recent interviews took place in 1987. For the purpose of the study, only 
the less educated were analysed, leaving a sample of 780. The dataset allowed the 
researchers to assess how both initial training and post-initial training had affected 
wages. However, a range of other dependent variables were looked at; for example, 
unemployment, job and function changes, motivation, attitudes to work and general 
career success were also considered. It was found that CVT increased interest in work 



The point is also made and developed empirically in Groot (1995). 

The selection issue is discussed in Booth (1991) and Lynch (1992), and Lynch estimates a model where 
a selection-correction procedure is used; however, the point is not stressed in either of these two studies. 
Another paper by Groot et al. (1994b) is concerned with the self-selection of individuals into employer- 
provided training programmes. 
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and also the willingness to change jobs. Again, this raises the issue of other dimensions 
of training effectiveness which could possibly be measured. 

Lynch, Lisa and Sandra Black (1995), "Beyond the Incidence of Training: 
Evidence from a National Employers Survey", National Bureau of Economic 
Research Working Paper No. 5231, Cambridge, Massachusetts. 

Black, Sandra and Lisa Lynch (1996), "Human Capital Investments and 
Productivity", American Economic Review, Vol. 86, No. 2, pp. 263-267. 

Black, Sandra and Lisa Lynch (1997), "How to Compete: The Impact of 

Workplace Practices and Information Technology on Productivity", National 
Bureau of Economic Research Working Paper No. 6120, Cambridge, 
Massachusetts. 

The estimation of the training/productivity relationship has recently been advanced in a 
series of papers by Lisa Lynch and Sandra Black (Lynch and Black, 1995; Black and 
Lynch, 1996; and Black and Lynch, 1997). As with many aspects of empirical research 
in the social sciences, their advances have been facilitated by a new dataset which, as 
the authors put it, “was designed to overcome some of the limitations of previous 
studies and collect more precise data on human-capital inputs and establishment inputs” 
(Black and Lynch, 1996, p. 263). The dataset derives from a survey from the National 
Center on the Educational Quality of the Workforce National Employers Survey (EQW- 
NES); through a phone survey in 1994 it generated responses from 1,621 manufacturing 
companies and 1,324 non-manufacturing companies. 

The authors use the data for a number of purposes. For current purposes, the first results 
of interest are found in the 1995 paper 4 . Production functions are estimated for the 
manufacturing and non-manufacturing sectors, in which dimensions of training are 
included along with the more usual factors in production functions such as capital and 
labour. The results on training are interesting; the number of workers trained is not 

4 The same results are found in the 1996 paper, a published version of a section of the 1995 working 
paper. 
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found to have a significant effect on productivity, but this masks the effects of different 
dimensions of training, which do matter. In manufacturing the higher the proportion of 
training that is off-the-job, the higher is productivity. Similarly, in non-manufacturing 
firms the type of training matters for the productivity effects; in particular, training in 
computer skills increases productivity. 

As the results presented in the 1995 and 1996 papers of Black and Lynch are based on 
data from a single year, they suffer from the problem discussed above of failing to take 
into account time-invariant unobservables. 5 In the 1997 paper, they attempt to overcome 
this problem by supplementing the EQW-NES data with data from the Longitudinal 
Research Database (LRD) of the United States Bureau of the Census. The authors were 
able to match the companies in the EQW-NES with records in the LRD and thereby 
create a dataset with over time information on the companies. In re-estimating their 
earlier work, they now find no effect of training on productivity; however, this might be 
because the information on training was too weak for its effect to be captured in the 
extended estimation framework. What does emerge from this study is the interesting 
effect that workplace practices have on productivity. In particular, greater involvement 
of workers in decision-making and the use of performance-related pay are seen to 
generate higher productivity relative to the more traditional labour/management 
relations. Such workplace practices will be discussed again below in the context of the 
paper by Ichniowski et al. 

Ichniowski, Casey, Kathryn Shaw and Giovanna Prennushi (1995), “The Effects of 
Human Resource Management Practices on Productivity”, National Bureau of 
Economic Research Working Paper No. 5333, Cambridge, Massachusetts. 

While this paper is limited in its references to training, it is nonetheless of interest both 
from a methodological point of view and for its results. The purpose of the paper is to 
assess the impact of human resource management practices generally, including 
training. However, the focus is not so much on the effects of individual policies; rather, 
an attempt is made to estimate the effects of collections or systems of policies. The 
theory underlying this approach is that human resource policies will have 



complementary effects: the use of certain policies in isolation will have a weaker effect 
than when such policies are combined with other human resource measures. 

In order to test the theory, the authors visited 26 steel plants in the United States and 
collected longitudinal data on workplace practices, productivity, technology and wages. 
They restricted their attention to a very specific process, thereby easing the comparisons 
across workplaces. The measure of productivity used was the proportion of “up-time” 
in production, i.e. the proportion of time during which the process was not stopped for 
whatever reason. They found that the hypothesis of complementarity among human 
resource measures, including training, was supported by their data. A similar finding on 
workplace practices can be found in Black and Lynch (1997), referred to above: they 
find that the effect of a human resource policy depends not so much on its introduction 
but on the manner of that introduction, i.e. whether other polices are introduced along 
with it. Such findings raise an important consideration for further research into the 
effects of training: if the effects are enhanced by the presence of other policies, it will 
be useful to structure the research so that this can be captured. 

Ichniowski et al. discuss another important issue which deserves our attention: having 
found that the introduction of human resource management systems produces 
productivity gains across the sample of firms, they ask why it is that not all workplaces 
introduce such systems. Their suggested answer centres on the indirect costs of 
introducing such policies. In particular, where workers resist doing things in a new way, 
the costs generated by of the upheaval caused may outweigh any long-term productivity 
benefits. This point is important in that while training may have positive effects on 
average, a firm’s decision not to provide training may not necessarily be due to its 
managers’ ignorance of training benefits. 



5 This is acknowledged by the authors; see p. 266 of the 1996 paper. 
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Ottersten, Eugenia Kazamaki, Thomas Lindh and Erik Mellander (1996), "Cost 
and Productivity Effects of Firm Financed Training", The Industrial Institute for 
Economic and Social Research Working Paper No. 455, Uppsala, Sweden. 

Ottersten, Eugenia Kazamaki, Thomas Lindh and Erik Mellander (1997), 
“Evaluating Firm Training Effects on Performance and Labor Demand”, mimeo. 

One of the few European econometric studies is that of Ottersten et al. (1996) 6 . The 
primary contribution of this study is to the theoretical modelling of the effects of 
training. Whereas all studies discussed to date have focused on the relationship between 
training and productivity, Ottersten et al. present a model of the relationship between 
training and cost reduction. There is, of course, an inverse relationship between 
productivity and cost, so while the focus may be different, the core issue is the same as 
in the studies already reviewed. The authors test their model using annual data from 
eight Swedish firms in the machine tool industry for the years 1975-1993; empirically 
they find large cost effects of training, although the effects as measured by productivity 
are smaller 7 . 

Von Bardeleben, R., U. Beicht and K. Feher (1995), “Betrachtung des Nutzens 
betrieblicher Ausbildung”, in U. Beicht, R. von Bardeleben and K. Feher (eds.), 
Betnebliche Kosten und Nutzen der Ausbildung. Ergebnisse aus Industrie , Handel 
und Handwerk, Bielefeld: Bertelsman 

Although this study was primarily concerned with training under the German dual 
system, and as such relates to initial training, it raises a number of issues which are 
relevant to this review, given that the training being studied is enterprise training. Two 
approaches were taken in considering the costs and benefits of training. A large-scale 
survey was conducted, in which 1,370 enterprises were interviewed. However, as it was 



6 Their 1997 study has a similar orientation. 

7 One other European study is that of de Koning and Gelderblom (1992). As their dataset is particularly 
small (50 companies), their finding of no impact of training on productivity is of limited use. Ottersten et 

al. have fewer firms in their dataset, but the availability of annual data for 18 years adds richness to their 
data. 
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believed at the outset that capturing the benefits of training would be difficult using this 
approach, case studies were also undertaken. 

While it was concluded from the case studies that it was extremely difficult to get 
reliable information on the returns to training in monetary terms, the survey included a 
number of questions which allowed for other effects of training to be identified. Firms 
were asked their reasons for training workers as opposed to hiring workers who are 
already trained, so in a sense the issue being considered is the cost-effectiveness of 
training relative to hiring those already trained. Among the reasons for training 
mentioned by a large proportion of firms were the following: 

1. To get skilled workers who could not be recruited from the labour market; 

2. To avoid high turnover, by tying the workers more closely to the firm; 

3. To be able to identify the best workers from among the trainees. 

The value of this study is to generate thought on what dimensions of training, other than 
productivity growth, may be measurable. 

Van der Sanden, and G.EJ.M. Hornman (1995), “Zelfstandig leren op de 

werkplek met behulp van de Leittextmethode”, in M. Mulder and W.S. de Grave 
(eds.), Ontwikkelingen in branche- en bedrijfsopleidingen, Utrecht: Lemma. 

The dimension of the “training-retums-estimation” issue to which this study draws 
attention is the potential for focusing on a very particular type of training. In the study, 
the ‘Leittext’ method 8 was being evaluated; this method aims to empower and 
encourage workers to learn and to work independently. The approach adopted was to 
compare two small groups of workers, those who underwent the ‘Leittext’ training, and 
those who did not. The methodology used is also instructive, in that the study gathered 
information on both the treatment and control groups both before and after the training. 



8 The Leittext method is an approach to training which aims to stimulate learning and working in a self- 
regulated way. Training constructed according to this method, for individuals or groups, usually entails a 
number of well-defined instructions. To perform each of these instructions, six different cyclical phases 
have to be passed: informing, planning, decision-making, executing, controlling and evaluating (and back 
to informing again, etc.). Each phase involves both theory and practice learning. 
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thereby creating the potential for measuring the net effects of the training. However, a 
general difficulty in this area was also highlighted; managers found it difficult to 
evaluate whether there was an increase in the willingness of workers to work 
independently. 

Von Bardeleben, R., U. Beicht, H. Herget, and E. M. Krekel (1996), Individuelle 
Kosten und individuelle Nutzen beruflicher Weiterbildung, Bielefeld: Bertelsmann. 

This study involved a survey in which reasons for training were sought. While some 
studies have asked firms their reasons (von Bardeleben et al., 1995), this study was 
concerned with individuals. This study aimed to assess the individuals’ views of the 
extent to which their goals in training had been realised. The main contribution of the 
study for current purposes is the identification of training effects other than the wage 
effect. Again like the previous study, the development of measures of training other 
than the wage effect may facilitate work in this area. Among the goals identified by the 
individuals surveyed were better work performance, more interesting or responsible 
tasks and increased mobility. One of the most interesting findings of the study was that 
higher wages ranked below a range of other training goals. If this finding generalises to 
other populations, it has important implications for which dimensions of training 
effectiveness we should be measuring. Another interesting finding was that the 
priorities which individuals held for training differed between Eastern and Western 
Germany; this indicates that societal, or more specifically, labour-market-related 
conditions impact upon the goals of training. 



Huselid, Mark A. and Brian E. Becker (1996), “Methodological Issues in Cross- 
Sectional and Panel Estimates of the Human Resource-Firm Performance Link”, 
Industrial Relations , Vol. 35, No. 3, pp. 400-422. 

In discussing the work of Holzer et al. above, we referred to the advantages of studies 
which present data over time on a cross-section of firms, i.e. a panel. Such datasets 
allow for changes in practices to be associated with changes in productivity, thereby 
avoiding upward biases in estimates where a correlation exists between the presence of 
training and unobserved firm characteristics. While Huselid and Becker accept this 
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advantage, they point out that in working with differences in values over time another 
difficulty is introduced into the analysis, that of measurement error. If recorded values 
of any independent variable are not entirely accurate measures of, for example, training 
input, but instead are subject to a random measurement error, the coefficient estimates 
associated with that variable will be biased downwards. If measurement error exists, the 
difficulties it creates are increased in differenced equations; the authors believe this to 
be a serious problem in accurately measuring the effects of human resource measures 
generally. 

An additional point made by Huselid and Becker relates to response rates for surveys 
used in this type of research. In their own study, they mailed questionnaires to 3,477 
firms in 1992 and 3,847 firms in 1994. However, response rates were only in the 20-30 
percent range. In addition, less than 10 percent of firms responded to both surveys, 
further restricting the panel estimates. Such low response rates inevitably raise 
questions about possible selection biases in responses, and while statistical techniques 
are available to correct for selection bias, the small sample sizes can reduce their 
usefulness. Huselid and Becker are of the impression that the lack of response arises 
because companies have a policy of not responding to surveys and/or because their 
workloads preclude participation. The authors comment that “such policies will only 
make survey research on this topic more difficult in the future”. In the short term, the 
lesson would appear to be that while studies based on samples with low response rates 
can yield insights, results must be interpreted and applied carefully. 

OECD (1996), Measuring What People Know: Human Capital Accounting for the 
Knowledge Economy , Paris: OECD. 

We include this study in the review not so much because of the methodology involved, 
but because of its general subject matter. The issue which this study investigates is 
whether human capital accounting is possible and feasible in theoretical and practical 
terms. 

Economists currently estimate human capital on the basis of schooling years or formal 
educational attainment levels, regardless of actual productive capacity. Financial 
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accounting and reporting mechanisms ignore even these crude measures and, due to 
certain conventions, omit the value of human capital from the balance sheet. 

A review of innovative policies in OECD countries shows that progress has been made 
in moving beyond the basic information provided in standard educational certificates. 
Encouraged by the notion of a ‘knowledge economy’, government policy makers, 
human resource managers, financial accountants and educationalists are developing 
methods for systematically evaluating and recording knowledge as assets acquired 
through experience, education and training. 

This study explains why it is possible in terms of economic theory, and feasible from the 
perspective of accounting practice, to implement new human capital information and 
decision-making systems. Reviewing the steps already being taken by individuals, firms 
and governments to improve the functioning of human capital markets leads to the 
conclusion that public policy must focus on the development of better signals for 
competence validation, valuation, accounting and financial reporting. Embracing a 
pragmatic approach, the policy directions suggested here build on the emerging new 
standards and institutions. According to this study, in the future the largest share of 
value-added in the economy as a whole will come from activities where people add their 
personal knowledge, insight and vision to a product (tangible or intangible). The radical 
break occurs when a large share of the economy shifts away from activities that are 
routine, repetitive and based on someone else’s instructions to production that is 
inventive, customised or personalised. 

Grunewald, U., D. Moraal, D. Gnahs, F. Draus and R. WeiB (1997), 
AbschluBbericht zum BMBF-Projekt Formen der ‘weichen Bildung’ (draft 
version). 

This study is concerned with the possibilities of generating quantitative information on 
what the authors refer to as ‘informal’ types of training or ‘work- integrated’ training. In 
the course of the study, the authors also identify a range of sub-categories within 
‘informal’ training. Broadly defined, they create sub-categories along the dimensions of 
off-the-job and on-the-job training and also along the dimensions of individual and 
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group processes. Hence, they produce a comprehensive range of types of training, 
although the distinction between them is not always as sharp as the classification may 
indicate. 

Having generated the categorisation, the authors reveal the problems which arise, as 
mentioned in interviews with enterprises, in trying to move toward a quantitative 
analysis of ‘informal’ training. These are (a) the difficulty of creating a demarcation 
between work and ‘work- integrated’ training and (b) the prohibitive time and energy 
input, as perceived by firms, in generating and gathering the information. 

Barrett, Alan and Philip J. O’Connell (1998), “Does Training Generally Work? 
Measuring the Returns to In-Company Training”, Centre for Economic Policy 
Research (CEPR), Paper No. 1879, London. 9 

Barrett and O’Connell (1998) use a dataset constructed over two points in time to 
estimate a training/productivity link. The first component of the dataset is a survey of 
Irish companies conducted in 1993 in which firms were asked for detailed information 
on their training activities. The second component of the dataset is a follow-up survey 
of the same companies conducted in 1997; this time the companies were asked for 
information on, among other things, output, capital stock and workplace practices in 
1993 and 1995. About 200 firms provided sufficient information for them to be used in 
the analysis. Given the way the data had been gathered, the authors were able to test for 
a relationship between the training input and changes in output and productivity. A 
finding similar to Black and Lynch (1996) emerges in that training itself is not seen to 
influence productivity; rather it is the type of training that matters. In the Barrett and 
O’Connell study, training that was described by the employers as being ‘general’ in 
nature (that is, useable elsewhere) increased productivity, but training that the employers 
classified as ‘specific’ (that is, not useable elsewhere) had no impact on productivity. 



9 A summary of this paper can also be found in CEDEFOP (1998), Approaches and Obstacles to the 
Evaluation of Investment in Continuing Vocational Training: Discussion and Case Studies from Six 
Member States of the European Union. 
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2.2 Conclusions 



Having presented the papers in a somewhat unstructured way, we now want to draw out 
the major themes and issues. Some of the studies can be grouped together because they 
attempt to establish a link between training and productivity using standard statistical 
techniques. Some have done this indirectly by establishing a link between training and 
wages and assuming that higher wages are paid when productivity is higher. Some have 
established the link directly by estimating links between training and productivity, 
defined in various ways. Other studies have looked at other training effects such as 
increased job satisfaction and reduced unemployment. All of the studies are very recent, 
reflecting that this is a new area of research. The reason for this is that, until recently, 
the datasets upon which the studies are based were not available. To a great extent, the 
quality of the results obtained are functions of the quality of the data with which the 
researchers worked. 

Although this review includes the word “returns” in the title, it is apparent from the 
above research that rates of return in the strict economic/financial sense of the word 
have rarely been estimated. What has been done is to relate training inputs, to the extent 
that the data allows such inputs to be measured, to training outputs, again in a manner 
that is dependent on the data available. We have not found many calculations which 
relate the initial investment in training to the flow of benefits over time, thereby 
producing an estimate of a rate of return in a manner which is done for investment in 
capital assets. Any such calculation would be subject to considerable uncertainty. This 
is partly due to data limitations, but also to the lack of good information on the speed 
with which the acquired skills depreciate or the extent to which employees change jobs. 
As the information on returns is so limited, it is not surprising that there is no 
information on how the size of the return might be related to the size of the training 
investment. As with many investment activities, it is likely that the returns to training 
investment start to diminish beyond a certain level. Such information would clearly be 
of great use in this area, although we appear to be a long way from obtaining it. 

One exception to this lack of rate-of-retum calculations is the paper by Mincer (1991). 
He gathers results from a range of studies and, imposing certain assumptions, estimates 
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a rate of return; for example, if a depreciation rate of 4 percent is assumed, returns 
ranging between 8.7 and 26 percent are found. However, in discussing whether there is 
evidence of under-investment in training, Mincer says the following: “..., there is no 
evidence of under-investment, though it clearly cannot be ruled out, given the wide 
range of estimates.” Hence Mincer, too, appears to say that we do not have what might 
be considered to be a reliable estimate of returns to training. 

While the research in this area may not have produced a reliable measure of the rate of 
return to training, it has nonetheless produced many interesting insights. In general, 
training is found to have a positive effect on wages (e.g. Lynch, 1992, and Booth, 1991) 
and productivity (e.g. Black and Lynch, 1996; Bartel, 1994; and Holzer et al., 1993). In 
addition, as Bishop (1994) found, the productivity effect is greater than the wage effect, 
thus making training a profitable activity for firms. 

Lynch (1992) also found that while off-the-job training with a previous employer 
increased wages with a current employer, on-the-job training with a previous employer 
did not. There are two important implications of this result. First, it is possible that 
current employers only reward previous off-the-job training because the content of such 
training is more transparent. If this is so, there is a reduced incentive to undertake on- 
the-job training for employees who believe they will change jobs at some stage. 
Second, the fact that current employers reward previously acquired off-the-job training 
implies that such training raises productivity in the current job. Hence, if this previously 
acquired training was financed by the previous employer, there is a “spillover” effect 
from one employer to another. Such an effect was also found by Bishop (1994), and so 
the possibility exists that there is a reduced incentive for employers to provide training. 
This, of course, is the familiar issue that if training is general in nature, in the sense of 
being applicable in other workplaces, employers may require the employees to finance 
the training, since the employer would not be in a position to appropriate any return on 
the training investment if the employee were to leave. 

Groot et al. (1994a) demonstrate the important point that in estimating the effects of 
training across a range of individuals, one must be aware that people self-select into 
training: hence estimates which do not take account of this may over-estimate the 



effects of training for some. Ichniowski et al. (1995) draw attention to this fact from the 
perspective of firms when they discuss why it is that some firms do not introduce 
training measures, even though such measures appear to have strong productivity effects 
when viewed across the sample as a whole. Statistical techniques do exist to overcome 
the difficulties associated with self-selection, but there can be limitations on their 
usefulness in certain circumstances. When self-selection corrections are not employed, 
care must be exercised in generalising results. 

Black and Lynch (1996) and Barrett and O’Connell (1998) found evidence to suggest 
that the provision of different types of training matter more for productivity than the 
provision of training per se. In Barrett and O’Connell, training that is general in nature 
is shown to be more effective than specific training in raising productivity. Black and 
Lynch (1996) find that in the non-manufacturing sector, the provision of off-the-job 
training is what matters. 

Other studies reviewed indicated the importance of combining training with other 
human resource measures in order for the training, and indeed the other human resource 
measures, to be effective (Ichniowski et al. 1995); clearly, this is an important point to 
keep in mind when trying to identify and measure returns to training. 

While the focus of much of the work is on measuring returns in terms of wage and 
productivity increases, some of the studies brought attention to other effects of training. 
For the enterprise, such other important effects may include reduced turnover (von 
Bardeleben et al., 1995). For the individual, the other effects may include reducing the 
probability of unemployment (Diederen, 1994), making work more interesting and 
increasing the possibilities of changing job functions or jobs (von Bardeleben et al., 
1996). 

The measurement of the effects of certain types of training may yield useful results, 
especially if there is a specific goal attached to that training, for example, encouraging 
independent learning under the ‘Leittext’ as discussed by van der Sanden and Homman 
(1995). Generally, the categorisation of training can also assist in return estimation 
exercises, although problems do remain; for example, it is not always easy in practice to 
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draw sharp distinctions between training types, or indeed between work and work- 
integrated training (Griinewald et al., 1997). 

Finally, although the issue of how to attach monetary values to dimensions of human 
resource investment runs through all the work considered here, we have seen how a 
pragmatic approach can be adopted, in which value ranges are used (van Sandick and 
Schaap-Neuteboom, 1993). We have also seen the beginning of a more systematic 
approach through the generation of more formal accounting procedures related to human 
resources (OECD, 1996). 

Drawing together these conclusions we can say the following: 

• strict rates of return, like those calculated for physical assets, have rarely been 
calculated 

• training is shown to have a positive effect on wages 

• training is shown to have a positive effect on productivity 

• there is evidence to suggest that training received from one employer increases 
productivity and wages with another employer 

• selection effects matter in training: i.e. the benefits are higher for those who choose 
to train compared to those who choose not to train, were they obliged to train. 

• different types of training matter for productivity effects, as does the combination of 
training with other human resource policies 

• individuals and firms have objectives for training other than wage and productivity 
growth 

Having reviewed many of the recent, prominent studies in the area and drawn 
conclusions, in the next section we will present the issues which arise in the estimation 
of returns to employer-provided training in a more general manner. Once again, our 
goal is not to present information as an end in itself, but rather to stimulate thought and 
discussion. 
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Having reviewed a range of studies and distilled the conclusions, we now alter our 
focus. In this section, we will discuss a range of issues which arise in measuring the 
returns to CVT investments. It should be noted that a number of these issues are closely 
linked and inter-dependent, and they should be viewed accordingly; nevertheless, we 
have tried to break them down as far as possible into discrete headings. In order to 
illustrate points, we will draw on the studies mentioned above and some additional 
references; however, our interest in this section is on general points and not on the 
specifics of each study. The headings under which our discussion will proceed are as 
follows: 

• Evaluating CVT returns at the level of the enterprise 

• Levels of evaluation 

• Types and forms of training 

• Types of returns 

• Who receives the returns 

• Methodologies 

3.1 Evaluating CVT returns at the enterprise level 

It is useful to consider the issue of CVT evaluation as it occurs at the enterprise level. 
This provides insights into a number of the general problems associated with evaluating 
training and gives some indication of what is required from social science researchers. 

Although many articles and books have been written on how to evaluate training, many 
authors conclude that, in practice, firms rarely undertake a systematic evaluation of 
training (Dougherty, 1992; Mulder, 1995). Dougherty suggests possible reasons for 
this: 

(i) The cost-effectiveness of the training may be so obvious that formal evaluation is 
unnecessary. Much employer-based training is so obviously cost-effective that it 



would be a waste of resources to undertake a formal evaluation. For example, some 
training results from the adoption of new technology, and is often provided by the 
vendor of the new technology as part of a package. In this context, it is difficult to 
isolate the productivity effects of the training from the effects of introducing the new 
technology in order to evaluate the cost-effectiveness of such training. 

(ii) It may be impossible or prohibitively expensive to obtain the data necessary for a 
formal evaluation of training. The economic evaluation of much employer-based 
training may not be a practical proposition for a number of reasons. First, it may not 
be possible to measure the productivity increase of the trainee. Much training is 
provided for workers who are not directly engaged in production, so any increase in 
productivity will be difficult to assess. In particular, it is difficult to undertake a 
convincing cost-effectiveness analysis of training provided for managerial and 
supervisory staff, which typically accounts for an important part of training. 
Secondly, it may not be possible to isolate the increase in productivity which can be 
attributed to training from the increase in productivity which can be attributed to 
complementary changes in the production process. Thirdly, it may be difficult to 
isolate the cost of training from the cost of production as skill development is often 
an inevitable by-product of the production process. And fourthly, even when it is in 
principle possible to make a formal evaluation of cost-effectiveness, the cost of the 
evaluation may render the exercise prohibitive (Dougherty, 1992, pp. 551-552). 

To the extent that firms do evaluate training, they seem to do so mostly in an informal 
and incremental way. Griinewald and Moraal (1996) suggest in their case studies a 
variety of ways to assess the success of training in firms, albeit work-integrated learning 
only. These are as follows: 

- the results of training can be examined and evaluated on the basis of key statistics or 
indicators, for example, number of complaints, delivery times, meeting delivery 
specifications, development of costs, illness of employees; 

- continuous discussion can be undertaken between personnel department/supervisors 
and management; 

- training results can be checked through giving trainees questionnaires, 
tests/examinations or through general monitoring; 



- in ‘quality circles’, and other management processes, an implicit evaluation takes 
place through the process itself; 

- testing the quality of products can be regarded as an evaluation instrument; 

- client satisfaction is an important success criterion, so contact with clients can be an 
appropriate instrument. 

These different means of assessing the results of work-integrated learning processes are 
used to varying degrees. Informal procedures dominate, and most enterprises use these 
to evaluate the success of training, rather than undertaking a strict cost/benefit analysis. 
Generally, if the objectives of the training have been seen to have been met, then this is 
viewed as sufficient evidence of a successful implementation of training measures. 

While these procedures may be in place, the fact remains that enterprises in general do 
not rigorously evaluate their training expenditures. This provides an opportunity for 
researchers, but it also presents a first obstacle. The opportunity relates to the 
information gap that clearly exists. The obstacle concerns data; without firm level 
evaluation, it is unlikely that the sort of information needed for a comprehensive 
research evaluation would be available. Hence, it seems that great advantage could be 
gained if firms and researchers would collaborate extensively in putting the evaluation 
of training onto a more systematic basis. The collaboration could occur in generating 
data, collecting and analysing it. We will return to this theme in our overall 
conclusions. 

3.2 Levels of evaluation 

One aspect of measuring the returns to CVT is to identify the level at which the 
evaluation should take place. This is an important issue when considering what 
outcomes should be addressed in order to measure the returns to training in future work. 
Hence, to put some order on this dimension of the overall issue, we present here a 
categorisation of levels. 

In 1959 Kirkpatrick - one of the founding fathers of research on returns to training - 
developed what is now considered the classic four-level approach to the evaluation of 
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training, which focused on: 



Level 1 - reaction - i.e. did participants adjust to/like it? 

Level 2 - learning - i.e. did participants learn? 

Level 3 - behaviour - i.e. did participants use what they had learnt? 

Level 4 - results - i.e. did the training fulfil its stated (or other) objectives? 

(Kirkpatrick, 1959). 

It should be noted that these levels do not represent mutually exclusive outcomes but 
rather they are different, and often inter-related, manifestations of the returns to training. 
For example, measuring level 3 (behaviour) will also provide a fair reflection of level 2 
(learning). This is not necessarily the case at level 4 (results), as this model can be 
viewed as combining two aspects of evaluating training returns: whether the training has 
been ‘effective’, i.e. did it meet its objectives, and whether the training has had an 
‘effect’ which may be linked to general objectives of the enterprise or employees rather 
than to pre-defined objectives specific to the training. 

Much of the research into the returns to training to date has focused on the first three 
levels of evaluation, given the number of data and methodological problems associated 
with level 4 (results) and the problem of identifying the objectives in order to develop 
appropriate indicators against which the training can be evaluated. As a result, much of 
this work has focused on the returns to the individual as an indirect, rather than a direct, 
indicator of the returns to the enterprise. Within this, research at level 1 (reaction) 
seems to be the most prevalent area of work (Dougherty, 1992; MacDonald, 1995; 
Mulder, 1995) within enterprises. Mulder (1995) states that most of the evaluation 
methods available are aimed only at observing the reaction of participants in training 
through surveying on the basis of ‘happy sheets’. He assumes a lack of methods and 
techniques for training evaluation beyond level 3, which can be used in practice, and he 
concludes that few practical instruments are available at the enterprise level to evaluate 
training returns in terms of overall work behaviour as it benefits the firm, due to the 
need to evaluate the returns to training in a specific way within different contextual 
environments. The perceived absence of level 4 (results) evaluation is also illustrated in 
the title of an article by Shelton and Alliger (1993); “Who’s afraid of level 4 
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evaluation?” 



Recent research which has focused on level 4 has done so in terms of direct ‘results’ for 
the enterprise (e.g. Brinkerhoff, 1988; Kessels, 1993; Bewyl & Geiter, 1996) on the 
assumption that the major reason for launching training activities derives from firm- 
level problems. A number of evaluation models at this fourth level are aimed 
principally at observations concerning the success of solving organisational problems 
through training (Kessels, 1993). Many of the studies in Section 2 highlight an 
increasing emphasis on level 4 evaluation from the enterprise perspective, which also 
indicates an increasing ability to overcome a number of the data and methodological 
problems (these are discussed in more detail below). 

Nevertheless, level 4 (results) evaluation remains hindered by the lack of an adequate 
definition of what constitutes training activity and the diffuculty of identifying what are 
the aims and rationale behind choosing particular types or forms of training as a means 
to evaluate returns against those aims and choices. 1 

3.3 Types and forms of training 

The work reviewed in Section 2 has already pointed to the need to differentiate between 
types and forms of training. Training is a generic term for a number of very different 
types of activities and also a fluid entity. Bentsson and Wurzburg (1992) indicate that 
the content of training is changing and being provided in a different way; this will 
clearly have an effect on the way the returns to such training can be measured. What 
remains unclear, however, is how training can be most usefully differentiated when 
estimating returns. As indicated by Tuijnman (1992): “continuing education is an 
elusive idea rather than a well-defined concept.” 

Various ways of categorising training forms are possible: e.g., on/off the job, 

formal/informal. As shown in Section 2, Griinewald and Moraal (1997) make a 



1 Further discussion concerning this issue can be found in an article by Sven-Age Westphalen 
“Continuing Vocational Training: from statistical necessity to analytical complexity” to be published in 
issue 51.1 of the Journal of Vocational Education and Tr ' g in 1999. 
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distinction between ‘off-the-job’ and ‘on-the-job’ training. Most of the work in this area 
focuses on ‘off-the-job’ training; returns to ‘on-the-job’ training are rarely considered, 
despite the increasing importance of ‘learning by working’ in modem societies. 
Exceptions are the studies of Griinewald and Moraal (1996) and of Ichniowski et al. 
(1995), reviewed in Section 2, and the CEDEFOP report on the learning effects of work 
organisation, done in various EU-countries (e.g., the Dutch study: Onstenk and 
Voncken, 1996). The lack of research into the returns to ‘on-the-job’/‘leaming by 
doing’ relates to the difficulty of making a distinction between working and training. 

In addition, training can be distinguished according to its content and aims: e.g. training 
for sales, training in computer skills, etc. Studies that have employed such a distinction 
have been able to generate interesting results on the relative impact of different types of 
training: e.g. Black and Lynch (1997) and Kirkpatrick (1994). 

One of the problems associated with distinguishing training on this basis is that, at the 
level of the enterprise, training objectives are not always clear. In addition, training may 
have multiple objectives which are not easily ranked. Only when the objectives of 
training are clearly stated can the type of training which should be evaluated against the 
objective(s) be identified. While a number of enterprises do formalise their training 
objectives (e.g. training in new technology, health and safety training, training in quality 
control, etc.), it is sometimes difficult to determine what the actual goals are, and even 
whether they are ‘real’. It is often the case that investment in training serves only the 
goal of legitimisation: i.e., investing to show that you are doing something about a 
problem/issue as it is defined by superiors, by law or by clients. 

Defining training activity and identifying and linking it with its or other more general 
objectives would take us one step closer to identifying the type of return against which 
the training should be measured and evaluated. 
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3.4 Types of returns 



The studies in this review indicate the difficulties which arise simply in defining returns; 
measuring those returns is even harder. This is partly due to the lack of training 
objectives discussed above. As a result, research into the returns to training cannot 
relate those returns to specific training objectives; instead the training is evaluated 
against more general objectives. For example, the returns to training may be measured 
in terms of productivity increases, but the training measure might include health and 
safety training, the objective of which is not necessarily an increase in productivity. 
Identifying returns in this way also assumes that the process within the type and form of 
training has been effective to meet the return criteria. Many of the studies reviewed link 
only the incidence of training, in terms of the volume of training activity undertaken by 
firms or on the participation rates of employees, with a general return. 

In addition, as with most returns, the returns to training can accrue in both ‘tangible’ and 
‘intangible’ terms, within which the demarcation line is not clear. With regard to 
tangible returns, a number of studies try to measure the benefits of training at the level 
of the enterprise in terms of productivity. The question arises, however, as to what 
measure of productivity should be considered: increased profitability, reduced costs, 
improved quality, reduction of faults, increased customer satisfaction, etc. As 
mentioned, this also implies that a certain measure of productivity can be affected by a 
variety of different training activities, and so it may be difficult to associate the precise 
training activity with its corresponding productivity effect. Kirkpatrick (1994) mentions 
exceptional situations in which this can be done; for example, increases in sales may be 
found to be directly related to a sales training program. As Kirkpatrick states, however, 
establishing a causal relationship between training activities and its effects is usually not 
possible. In this respect, both the lack of strict definitions in this field and the numerous 
objectives which training has for the different stakeholders impact greatly upon our 
ability to establish causal relationships. 



For measuring intangible returns, tangible indicators have to be generated; this often 
involves making assumptions which are hard to verify. For example, an enterprise that 
tries to improve the social and communicative skills of its employees through training 
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might use the turnover rate of both training participants and non-participants as an 
approximate measure to investigate this particular effect of the training. However, such 
a proxy measure does not necessarily indicate whether the training itself has led to the 
desired or to the opposite ‘return’. 

The studies reviewed in Section 2 indicate a variety of returns that have been measured 
for both enterprises and individuals. For enterprises, evaluation tends to concentrate 
both on productivity and on the performance of participants: for example, Holzer et al. 
(1993) use a reduction in the scrappage rate; Barrett and O’Connell (1998) use a sales- 
based measure of output per worker; and Bartel (1994) uses a value-added-based 
measure of output per worker. Some larger companies, as described in Kirkpatrick 
(1994), have attempted to take into account direct returns at the organisational level 
(sales, turnover rates). Other, indirect returns (such as a reduction in recruitment costs) 
are not so well examined in the empirical studies found. Von Bardeleben et al. (1995) 
focus on this issue, but their research concerns initial vocational training in the dual 
system. 

For the benefits which accrue to individuals, a number of studies measure the wage 
gains of participants and promotion opportunities as types of returns. Other kinds of 
studies - more sociologically oriented - in this area identify several types of tangible and 
intangible returns; some of these have been reviewed in Section 2 of this report. As an 
example of the non-wage return, one can consider the work of von Bardeleben et al. 
(1996) in which they look at increased job satisfaction and the increased chances of 
changing jobs. 

To a large extent, the type of returns which are examined are a function of (a) the 
objective of the evaluation, and (b) the data which is available concerning the training. 
In addition, there is the issue of ‘perspective’, i.e. how the returns are appropriated for 
the different stakeholders. 
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3.5 Who receives the returns 



One of the reasons this review was undertaken is the lack of information concerning to 
whom the benefits of training accrue: i.e., which perspective is most important when 
estimating how the benefits of enterprise training are appropriated between the different 
stakeholders? This issue is important for a number of reasons. If certain actors are in a 
strong position to appropriate the gains from training, they are also more likely to 
finance the training. Hence, if it is shown that firms appropriate significant gains, then 
one can assume that profit-maximising firms will provide such training. However, if 
firms are unable to appropriate the gains from certain types of training, such as training 
that is general in nature (following the Becker definition), they may fail to provide that 
type of training. Again, we discuss this issue in a general way, concerning what has 
arisen from the studies reviewed in Section 2, although we recognise that there are 
significantly different funding arrangements for enterprise training in different countries, 
which may enhance the relevance of certain returns from different perspectives. 

The review in Section 2 indicates that research has been undertaken on the returns for 
individuals and for firms. One of the areas where research is particularly 
underdeveloped is returns to the state, 2 either in economic or social terms, and to 
individual sectors of the economy. Documents and literature are available about the 
potential benefits for the state of encouraging CVT investment, but these tend not to be 
empirically based (Kath, 1994). 

Although no studies which attempted to estimate the benefits to the state of employer- 
provided training have been found, it is nonetheless of interest to consider here what 
these benefits might be and whether they can be measured. Possibly the most obvious 
return to the state is the increased tax revenue that arises from training which enhances 
productivity. Such productivity yields benefits to both the state and the enterprise if the 
trained workers remain within that enterprise. An increased number of trained workers 



2 Perhaps some exceptions can be found in Coopers & Lybrand (1996) and Denys & Aalders (1996), although this 
work contains many methodological limitations. Whilst not within the scope of this review study, the returns to 
training the unemployed for the state are more easy to quantify than the returns to training employees by firms, e.g. in 
terms of employment or unemployment rates. This can be illustrated in the Netherlands by the evaluation research 
done by NEI into the ‘net-effectiveness’ of training courses paid for by the Employment Services (de Koning, 1991). 
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may also lead to increased foreign investment which may also benefit the state - 
although not necessarily the training enterprise if their workers move. 

In addition to providing direct revenue benefits for the state, it can also be argued that 
training can reduce state expenditure. This can arise, for example, if employer-provided 
training leads to a reduced incidence of unemployment and, hence, reduced spending on 
unemployment and social benefits. While such arguments can be used to justify state 
intervention into the provision of continuing vocational training, the complexity of the 
issues makes it difficult to identify the way in which the state appropriates its returns to 
enterprise training. 

In addition, the returns to enterprise training organised at the collective (sectoral) level 
have rarely been considered. This type of training can be termed ‘sectoral training 
policies’, paid for by social partners at branch - or sectoral - level as a way to share and 
mutualise the training costs for enterprises in the branch and to reduce or avoid under- 
investment in training by firms (van den Tillaart and Warmerdam, 1995; Denys and 
Aalders, 1996). 

While sectoral training policies differ in the way they are implemented, the aims of the 
training do not differ from that of policies undertaken by individual firms. In terms of 
the returns, however, at least two different dimensions can be identified: a) the 

comparison between individual firms within the sector which implement training and 
those that do not, and b) the extent of success in avoiding both free-rider behaviour and 
under-investment in training by enterprises. Concerning a), only one empirical study is 
known; it considers the implementation of training in three different sectors through 
case-studies (de Vries and Hovels, 1991). Concerning b), no empirical studies exist, 
although the social partners seem to find it worthwhile to maintain sectoral training 
policies and finance them from collective bargaining sources. One of the major reasons 
that the returns to sectoral training provision have attracted little attention in empirical 
research seems to be that it occurs in only a few countries (e.g. Netherlands, Belgium, 
France and Denmark). 
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Yet, given that the nature of the employment relationship might be changing through an 
increase in short-term and part-time contracts, where employers might be less concerned 
with training their existing workforce and employees are more transitory, it is perhaps 
more likely that the responsibility for continuing training might increasingly fall to the 
individual or the state (this is already the case in some countries). This trend may also 
be seen in relation to the growth in the number of small- and medium-sized enterprises, 
where the provision of training is disproportionately expensive. Such a situation 
reduces the relevance of considering returns from the individual enterprise perspective 
and suggests that returns to whole sectors or to a collection of firms of a particular size 
should be considered, as well as the returns to state funding, collective funds, and 
individual funding. 

In addition, little research has been undertaken to consider the returns to training from a 
multi-stakeholder perspective as a means to understand more fully how the returns to 
training are appropriated. From the studies reviewed in Section 2, only one (Bishop, 
1994) provides an indication that both the enterprise and the individual appropriate 
some of the training return. 

One of the problems with identifying who appropriates the returns to training is 
associated with defining what constitutes a return from the different perspectives. To 
clarify this, we can assume that there are certain returns to training which are mutually 
beneficial to the individual and the enterprise; for example, research at level 3 
(behaviour) tends to focus on the ‘adaptability’ of training within the workplace (e.g. 
certain dimensions of this are explored within studies by den Ouden, 1992; Gielen, 
1995; Simons, 1992). The ability to utilise and ‘adapt’ training to the workplace can be 
a benefit both for the individual and the enterprise; the individual may be able to achieve 
higher levels of job satisfaction, wages, etc., while the enterprise will gain the ‘return’ 
through ensuring that a new or updated skill is gained. To some extent, both the 
individual and the enterprise will appropriate some of the training ‘return’. 

However, another dimension of training ‘adaptability’ is training ‘transferability’, where 
the skill gained or updated by the individual can be transferred to another enterprise. In 
this situation, there is the potential for the individual and the receiving enterprise to 
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appropriate more of the training ‘return’ than the enterprise that originally sponsored the 
training, i.e. the ‘spillover’ effect discussed earlier. 

The different perspectives associated with training evaluation confound a number of 
other related issues, not least the type of returns which might be evaluated. For 
example, a study undertaken by Griinewald and Moraal (1996), while concerned with 
training evaluation at the level of the enterprise, found that different stakeholders within 
enterprises use different criteria and parameters for evaluating the returns to CVT. 
Training participants and training staff viewed learning results (level 2) as the most 
important indicator for training effectiveness; participants also viewed the effect of 
training on working behaviour (level 3) as an important criterion. For management 
personnel, however, the results at the firm level (level 4) were the most important 
definer of training effectiveness. In addition, some of the returns measured can be 
viewed as secondary to the beneficiaries; for example, the study by von Bardeleben et al. 
(1996) indicated that, for individuals, higher wages are not always the primary return 
from training. 

However, von Bardeleben et al. (1996) and Griinewald et al. (1997) also indicate an 
overlap between certain returns and how they are perceived by organisations and 
individuals. This concerns certain specific, intangible returns: in particular, motivation 
of employees and better opportunities for workers to do their work properly. The latter 
type of return seems to be as important to the organisations as it is to individuals. 
Houman-Sorensen (1997) also indicates areas where goals are increasingly overlapping, 
such as enhancing productivity, mobility and employment. 

Regarding the possible different perspectives within the enterprise, Van der Krogt 
(1995) has developed a model which considers the integration of work and training 
types, and within that, the different strategies of the different stakeholders. This work, 
amongst other things, highlights the need to create frames of reference for the different 
stakeholders rather than define one objective or return against which to measure training 
effectiveness. 
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This has implications for the direction of future work: researchers need to ensure that 
they are asking the right questions of the appropriate stakeholders concerning the returns 
to training. In addition, it could be useful to try to develop work along a multi- 
stakeholder basis in order to establish which objectives underly training activity from 
different perspectives and how varied types and forms of training lead to different 
returns and benefit appropriations. This presents a formidable challenge in terms of 
collecting data and formulating an appropriate methodological approach, a more general 
problem in this type of work. 

3.6 Methodologies 

Both quantitative and qualitative research methodologies have been used in this area. In 
addition, a number of case studies appear to use - sometimes exclusively - quantitative 
research tools as well (e.g. van Sandick and Schaap-Neuteboom, 1995). 

Experimental research (where a group of workers within an enterprise are randomly 
assigned and trained for the sole purpose of evaluating the effects of training) in this 
area, while supported strongly in many conceptual studies, is rare. As a result, most 
studies “are likely to have suffered from variable sampling error and specification error 
resulting from selection bias, since only participants are investigated” (Tuijnman, 1992, 
p. 520). Quasi-experimental approaches were used, amongst others, by van Sandick and 
Schaap-Neuteboom (1995), van der Sanden and Homman (1995) and some large 
companies described in Kirkpatrick (1994). Experimental research appears difficult for 
a number of reasons. Several studies mention the impossibility of establishing a control 
group in a company setting, due to both practical and ethical problems. In addition, 
experimental research is expensive and more frequently used in studies concerning the 
effect of state-provided job-training. One view is that experimental research - however 
useful it may be in eliminating the influence of ‘disturbing variables’ (e.g. selection 
effects) and in estimating the ‘true’ contribution of training to returns - also has its 
limitations, as it usually renders little information on the training processes that 
determine the impact of training. 
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More common than experimental research are the evaluation studies that compare the 
‘post’-training situation with the ‘pre’ -training situation, where employees have been 
assigned to training in the usual, rather than in a ‘random’ way. Much of this research 
simply relates inputs to outputs with little emphasis on the ‘process’. Within this the 
outputs are also ‘uncertain’ in that the training goals are often unclear. As a result, the 
wrong type of output may be measured. In only a few of the studies are the goals of 
training pre-defined in such a way that the training results can be compared to these 
goals. In particular, surveys that concern employees in more than one organisation often 
lack information on the goals of training. The study of von Bardeleben et al. (1996) is 
an exception in this respect, in that it focuses on the balance between costs and benefits 
of training in a ‘qualitative’ manner, i.e. regarding the subjective experiences of 
participants. 

The methodologies used are often determined by the type of CVT activity being 
examined. Studies on work-integrated learning (Griinewald et al., 1997) use case- 
studies and qualitative techniques (e.g. interviews). Griinewald et al. believe that a 
qualitative approach for this type of training seems the only appropriate one, given the 
complexity of the issues and the difficulty of generating quantitative measures. Studies 
evaluating ‘off-the-job’ training often use quantitative techniques because this type of 
training is more clearly defined and hence is more quantifiable. 

Many handbooks 3 describe the research tools that are considered appropriate for 
evaluating each of the four levels of Kirkpatrick. In general, the research techniques 
undertaken in any given situation are partly determined by the level of evaluation being 
examined. Level 1 tends to be assessed through questionnaires; levels 2 and 3 by tests, 
questionnaires and interviews. Evaluation at level 4 involves not only the most 
difficulties, but also the greatest variety in research techniques used: questionnaires and 
interviews as well as quantitative measures of training outputs, e.g. productivity. 

With respect to econometric studies which use large-scale datasets, certain 



3 For example, Craig, Robert L. (1996), The ASTD Training and Development Handbook: a guide to 
human resource development , 4th ed., New York: Mcgraw-Hill; Philips, Jack J. (1997), Handbook of 
Training Evaluation and Measurement Methods, 3 rd ed., Houston: Gulf. 
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methodological problems should be noted; they refer to self-selection, low response 
rates in surveys, and bias both in cross-sectional studies (due to unobserved firm 
characteristics) and in panels (due to measurement error). The studies highlight a 
number of these issues. Such problems are compounded when undertaking research on 
a cross-firm, cross-sectoral and cross-national basis. Even at the firm level, different 
governing strategies and approaches will make comparative work difficult. A number 
of different factors affect a firm’s human resource strategies; such factors and the 
balance between them will alter according to the nature of that firm. These research 
problems and potential errors become more pronounced at the macro-economic level. 

While issues such as selection effects raise difficulties for quantitative approaches, these 
difficulties are not entirely insurmountable; the real challenge is to design a research 
strategy that reduces as far as possible any problems associated with the data or their 
analysis. Since low response rates are a problem which plagues work in this area, it is 
very important that efforts be made to ensure the highest possible response rate. The 
use of personal interviews rather than mail- or phone-based questioning usually leads to 
a higher response rate. Such an approach is also more expensive, so a trade-off must 
bemade. It can also be helpful for researchers to enlist the support of groups such as 
trade associations in an effort to generate higher response rates. 

With regard to the comparability of data across firms and across countries, careful initial 
planning is again required. One way to enhance comparability is to study similar firms; 
however, a trade-off must again be made because the more limited the group of firms 
being studied, the harder it is to generalise from the results. For this reason, it is 
preferable to ensure that the data collected across firms is as comparable as possible. As 
all firms collect data on profit, sales, etc., as part of usual business practice, this kind of 
information can be used, as it is by Barrett and O’Connell (1998) and Black and Lynch 
(1997). But it should also be possible to ask firms to collect information on issues such 
as staff turnover, which are again comparable. As with the low response rate issue, the 
challenge for the researcher is to come up with innovative ways of overcoming 
obstacles. 
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Finally, on the issues of self-selection and cross-sectional bias, some of the studies 
reviewed indicate how these difficulties can be overcome through the use of techniques 
of first-differencing and selection-correction. As Huselid and Becker (1996) point out, 
other problems can arise when efforts are made to address one econometric difficulty. 
But again, we would argue that this does not mean that efforts should stop; instead, 
greater efforts should be made to advance knowledge. The solution to some 
econometric difficulties can sometimes be found in the data-collection process; 
solutions to other difficulties require advances in technique. In the meantime, the 
existing tools provide the best means available to us and they should be used with care 
and honesty, in the sense that result write-ups should advise caution when there are 
reasons to believe that econometric results might be contaminated in some way. 

It is clear, therefore, that different types of CVT and different levels of evaluation as 
well as the different ‘return’ perspectives have a significant influence on the method 
employed. In addition to this, the methodological approach is also influenced by the 
data which is available or can be collected. 
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4.1 Conclusions 



The following quote from an education and training manager of a large company is cited 
in Kirkpatrick (1994): “We know that training pays off. Never doubt that. The dollar 
return is there, and it is high.” To a great extent, the central issue of this review is 
whether or not there is hard scientific evidence on this point and how additional 
evidence can be generated. It is true that many of the studies reviewed found positive 
effects of training. However, very few of the studies have linked the benefits back to 
costs in a way that allows us to think in terms of a rate of return. Hence, the question 
still remains: at the ‘margin’, should business be spending an extra ECU on training, or 
on physical capital, marketing, etc.? 

It can be concluded from our review that the amount of empirical research concerned 
with estimating the returns to CVT is rather sparse and underdeveloped, relative to other 
areas of investigation. Insofar as studies at level 4 (results) have occurred, their focus 
has been on individuals and enterprises, rather than on the state or sectors/branches. 
Within this, very few studies which try to apportion returns across the stakeholders have 
been identified. Furthermore it can be stated that almost all studies concentrate on 
‘formal’ forms of training, although it is informal training which is rapidly growing 
within firms. 

Within this latter issue is the problem of identifying what the original goals of the 
training were in order to establish a reliable return indicator. Many of the studies have 
linked the incidence of training to a certain return indicator which may be inappropriate: 
while it may show that the training had a positive ‘effect’ , it may neglect the issue of 
whether the type and form of training was an ‘effective’ way to meet specific goals. 
Identifying appropriate return indicators is also complicated by the fact that there are 
tangible and intangible returns to training. 
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In addition, it seems that various actors have different criteria for returns and evaluate 
those returns in a different way; for example, while an employee might see training as 
being transferable, an employer might see the same training as likely to generate a 
spillover effect. This creates a number of problems in assessing the returns with respect 
to the way in which the benefits are appropriated. 

A final conclusion can be drawn in that the findings of the existing studies in this area 
are very hard to compare with each other due to the diverse nature of the work. Some of 
them are conducted within firms, while others are conducted across firms; some are 
conducted by enterprise staff-members, others by universities or research institutes with 
rather different points of view on training. Furthermore, some of the studies are of a 
purely qualitative character; others encompass qualitative and quantitative research, 
considering more intangible and/or subjective aspects within big-scale surveys, be it 
cross-sectional and/or longitudinal on a cohort-basis. 

It is somewhat utopian to think that a single definitive number can be produced 
regarding the returns to continuing training within enterprises for the different 
stakeholders, although this does not undermine the need for further work. As 
Kirkpatrick states: “Be satisfied with evidence if proof is not possible” (Kirkpatrick, 
1994, p. 68). The work reviewed does indicate what might be achieved, at least from 
the enterprise perspective. For example, the work that shows higher benefits flowing 
from training when it is used in conjunction with other human resource policies 
(Ichniowski et al. 1995) provides an insight of practical use. In thinking about what 
might be done next, we can usefully ask the question, what would business/policy- 
makers/individuals like to know that (a) we currently do not know and (b) we might, as 
researchers, be able to answer using existing techniques. 

From the discussion in Section 3, there are certain issues which need to be considered in 
formulating further work in measuring the returns to enterprise CVT, these include: 

The effects of training which can be viewed as occurring in an immediate fashion, 
such as through participant’s reactions, or in a more distant way, such as through 
changed behaviour or increased productivity. The issue has to be addressed of 
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what level of evaluation is most useful, given the very different reasons for which 
enterprises implement training, and whether level 4 evaluation should be 
supplemented in a way that provides a greater insight into the training process and 
its effectiveness, as well as the end results. 

(ii) The different types and forms of training (formal/informal, specific/general, etc.), 
especially given that the boundaries of work and learning might be blurring. 
Some of the studies reviewed in Section 2 indicate that work is progressing in this 
area: e.g. the findings of both Black and Lynch (1996) and Barrett and O’Connell 
(1998) show that productivity is affected differently according to the type of 
training pursued, rather than the overall provision of training per se. 

(iii) The types of returns and how these relate to training objectives or other more 
general objectives. Within this is a need to generate measures of intangible as 
well as tangible returns; for example, for individuals we have noted that increased 
job satisfaction may be more important than the wage effects of training. It is 
therefore important to generate measures of these types of intangible returns. 

(iv) The training beneficiaries - individuals/firms/sector/state - and the way in which 
the benefits are appropriated between the stakeholders. We know from the 
discussion above that work is particularly underdeveloped concerning the state, 
for example. Inherent to the appropriation of benefits is the perspective from 
which returns are measured, given that the different stakeholders may have 
different criteria for what constitutes a return, for example, training transferability 
from the individual perspective versus the ‘spillover’ effect for enterprises. On 
the other hand, for certain returns there may be agreement as to what constitutes a 
mutual benefit. 

(v) The different methodologies and research disciplines (qualitative/quantitative, 
economic/sociological, etc.). Inherent to this point is the availability of data from 
enterprises or the ability to generate the necessary data. 

Whether some of these issues should be addressed independently - as subsidiary 
questions to the overall issue of evaluating work on the returns to continuing training in 
enterprises - or treated within a comprehensive framework is partly dependent on how 
work should progress. To a large extent, there is a trade-off concerning what can be 
achieved in the short and the long term. 





4.2 Further work 



In the short term, a number of benefits could be gained by investigating certain 
subsidiary questions. Some of these have been addressed in the studies discussed 
above, but the work remains fragmentary and could be supplemented. Among the 
questions which could be addressed are the following: 

• Why do enterprises implement training? 

• What types of training are most useful and for whom? For example, should 
companies bother providing training of a general nature, or should they restrict their 
efforts to training that is specific to their needs? 

• How does the effectiveness of training depend on the implementation of other human 
resource strategies? 

• Does the provision of training have an impact on productivity through its effect on 
other variables? For example, do workers who receive training remain with the firm 
for longer periods? 

• Reversing the above item, are trained workers more likely to leave? In particular, is 
the theoretical prediction that workers who receive general training are more likely to 
leave seen in real life? And if such workers do leave, what does this imply about the 
returns to training provision for the firm? 

• Does training received elsewhere raise productivity on the current job? 

In addition, as a general recommendation, more thought should be given to 
methodological issues, for example, the suitability of qualitative and quantitative 
research and whether they can be used in a complementary way, the definitions of 
training and ‘returns’ and other such issues, which are important in designing research. 

In the longer term, a more coherent approach could be developed alongside other 
advancements in evaluating enterprise training investment, such as Human Resource 
Accounting. In addition, the development of a more comprehensive stakeholder-based 
approach and combining expertise from a number of different disciplines and research 
methodologies could advance future work. 
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One example of such a project could be a collaborative exercise on CVT evaluation 
between the research and business communities. Given that there is a lack of formal 
evaluation of training undertaken at enterprise level, this clearly could yield mutual 
benefits for both groups. To be effective, such a collaborative exercise would 
commence with research design and would continue through data collection, analysis 
and presentation. Researchers could assist companies by identifying the information 
which they should be gathering and subsequently using to evaluate CVT investments. 
To be successful, the research/business partnership would need to exist over a period of 
time and not just when the researchers arrive to conduct a one-off interview. Ideally, the 
researchers’ involvement in the training evaluation should begin before a training 
investment is undertaken. In this way, the researcher can contribute to the establishment 
of baseline indicators such as output per worker, amongst others. Once the training is 
completed, the baseline indicators could then be used to provide some insights into the 
returns to the training investment. In this way, the analytical talents and academic 
interests of the researchers are combined with the information needs of the enterprise. 

The objective of this review has been to outline the issues involved in measuring the 
benefits of employer-provided training and to stimulate thought and discussion on how 
research in this area might proceed. It is clear that there are many difficulties in 
approaching this topic and so there may be a reluctance to confront the issues. 
However, we would argue that advances in our knowledge in this area are crucial if 
optimal resource allocations are to be made in the area of training and human capital 
development generally. 
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Exploring the returns to continuing vocational training 
in enterprises 

A review of research within and outside of the European Union 



Summary 

Inherent in the notion of lifelong learning is that enterprises should 
increase investment in continuing vocational training. The general 
lack of information, however, as to the returns of such training for the 
State, sectors, enterprises, and individuals means that there are a 
number of difficulties in evaluating current and future investment 
decisions. This review presents research which has attempted to 
estimate the returns of employer-provided training, discusses the 
conceptual and methodological issues which exist in this context, 
and suggests ways in which research in this area can be usefully 
expanded. This report represents the first stage in formulating a 
CEDEFOP project in this area. 
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